The molecular determinants for the activities of the reported benzoic acid (SH4-54), salicylic acid (BP-1-102), and benzohydroxamic acid (SH5-07)-based STAT3 inhibitors were investigated to design optimized analogues. All three leads are based on an N-methylglycinamide scaffold, with its two amine groups condensed with three different functionalities. The three functionalities and the CH2 group of the glycinamide scaffold were separately modified. The replacement of the pentafluorobenzene or cyclohexylbenzene, or replacing the benzene ring of the aromatic carboxylic or hydroxamic acid motif with heterocyclic components (containing nitrogen and oxygen elements) all decreased potency. Notably, the Ala-linker analogues, 1a and 2v, and the Pro-based derivative 5d, all with (R)-configuration at the chiral center, had improved inhibitory activity and selectivity against STAT3 DNA-binding activity in vitro, with IC50 of 3.0 ± 0.9, 1.80 ± 0.94, and 2.4 ± 0.2 μM, respectively. Compounds 1a, 2v, 5d, and other analogues inhibited constitutive STAT3 phosphorylation and activation in human breast cancer and melanoma lines, and blocked tumor cell viability, growth, colony formation, and migration in vitro. Pro-based analogue, 5h, with a relatively polar tetrahydropyranyl (THP) ring, instead of the cyclohexyl, showed improved permeability. In general, the (R)-configuration Pro-based analogs showed the overall best profile, including physicochemical properties (e.g., microsomal metabolic stability, Caco-2 permeability), and in particular, 5d showed improved tumor-cell specificity.